Artificial neural networks (or simply "Neural Networks") and their applications are well known in the art. "Neural" comes from neurons, or brain cells. Neurons are built by imitating in software or silicon the structure of brain cells and the three-dimensional lattice of connections among them. Another technique utilizes mathematical algorithms or formulas to accomplish pattern recognition. Neural networks have a remarkable ability to discern patterns and trends too subtle or too complex for humans, much less conventional computer programs, to spot. Neural networks can perceive correlations among hundreds of variables, recognizing patterns, making associations, generalizing about problems yet to be experienced before, and learning by experience. Ordinarily, computers mechanically obey instructions written with uncompromising precision according to a set of rules; however, neural networks learn by example. For example, to teach a computer associated with a neural network to tell the differences between good or bad symbols, patterns, characters, etc., examples must be learned by the neural network. Once the neural network has seen enough examples, the neural network can direct another device to react appropriately to subsequent examples.
In an article by Richard P. Lippmann entitled "Neural Nets for Computing", IEEE International Conference on Acoustics, Speech, and Signal Processing, 1988, three feed-forward models are described. Single and multi-layer perceptions which can be used for pattern classification are disclosed by Lippmann, as well as Kohonen's feature map algorithm, as described by Kohonen et al. in "Phonotopic Maps--Insightful Representation of Phonological Features for Speech Representation," Proceedings IEEE 7th International Conference on Pattern recognition, Montreal, Canada, 1984, which can be used for clustering or as a vector quantizer.
U.S. Pat. No. 5,245,697, granted to Suzuoka describes a neural network processing apparatus that calculates average absolute values and utilizes the resulting average differences between the output values and a center value to set each neuron to a controlled function which allows the neuron network to correctly identify an unknown multivalued image pattern.
U.S. Pat. No. 5,222,194, granted to Nishimura, describes another method of modifying neuron weights and reaction coefficients by changing input and output characteristics of units and weights of links in accordance with outputs of the output layer and a particular rule.
U.S. Pat. No. 5,212,767, granted to Higashino et al., describes a multi-layer neural network having input, hidden and output layers and a learning method wherein a processor belonging to the hidden layer stores both the factors of multiplication or weights of links for a successive layer nearer to the input layer and the factors of multiplication or weights of links for a preceding layer nearer to the output layer. Upon forward or backward calculation, the access to the weights for successive layers among the weights stored in the processors of the hidden layer can be made by the processors independently from each other.
U.S. Pat. No. 5,195,169, granted to Kamiya et al., describes a control device for controlling the learning of a neural networks wherein weight values of synapse connections between units of the neural network are monitored during learning so as to update the weight values.
U.S. Pat. No. 5,214,747, granted to Cok, describes a neural network wherein a memory array stores the weights applied to each input via a multiplier, and the multiplied inputs are accumulated and applied to a lookup table that performs threshold comparison operations.
Neural networks have been joined to standard large optical character recognition systems to improve their recognition. Older systems used for automated processing of forms and documents were usually limited to reading typed or clearly printed numbers and block letters. Neural networks improve accuracy and recognize diverse characters. Neural networks have been used extensively for character recognition in both font specific and omni-font classifiers. Generally, character classification can be considered as a partition of the feature space into several subspaces, each of which corresponds to a particular character or common character feature. This is illustrated in FIG. 1, where the feature space is two-dimensional. In this example, two characters, say "A" and "B" (represented by small circles) are to be recognized. The two large circles represent various features of different "A"s and "B"s. The dashed line is the partition of the space of the classifier. Overlapping of the larger circles implies that certain character samples are indistinguishable in this 2-D feature space. Consequently, even a perfect classifier will make errors. The characters from a single document are likely to be from one or a few fonts, and have similar distortions. As a result, the characters from a single document tend to cluster in feature space, as shown in FIG. 1 by small circle "A".
While the prior art provides for character recognition, the art does not provide a means for reducing classifications errors during character recognition. Therefore, there is a need for a system which reduces classification errors during character recognition using neural networks. There is also a need for a system which reduces classification error during recognition of characters, patterns, symbols, or any other medium well known in the neural network art.
Therefore, the object of this invention is to provide a system and method that can reduce classification errors by temporarily re-adjusting the classification boundaries during character recognition or the recognition of other medium. It is further object of the invention to provide a means and method that can take advantage of on-line unsupervised neural network learning techniques to adjust the classification boundaries during the recognition process.
It should be appreciated that neural networks have many applications in the character, object, symbol, or pattern recognition art. The following disclosure will refer to all possible medium as "characters" which should not be taken in a limiting sense with regard to the applicability of the teachings herein. All of the references cited herein are incorporated by reference for their teachings.